Teacher Education Quarterly, Spring 2010 


Engineering a Dynamic 
Science Learning Environment 

forK-l 2 Teachers 

By Ratrida L. Hardre^ Mark Namy, 

Hazen Refai, Chen Ling & J anis Slater 


Patricia L. Hardre is 
an associate professor 
in the Department of 
Educational Psychology 
and associate dean for 
graduate programs and 
research with thej eannine 
Rainbolt College of 
Education, M ark Nanny 
is an associate professor 
of engineering, Hazem 
Refai is a full professor 
of engineering, Chen 
Ling is an assistant 
professor of engineering, 
and Janis Slater isa 
graduate student, all at the 
University of Oklahoma, 
Norman, Oklahoma. 


Teachers are eager learners and participants when 
professional development opportunities are relevant to 
thei r personal i nterests, cl assroom needs, or both (B asi sta 
& M athews, 2002; Gordon, 2004). However, itisoften 
difficult for teachers to translate professional develop- 
ment concepts to thecl assroom i n a way that meets thei r 
students' needs (Duffy, 2004; Gordon, 2004). In-depth 
research on teachers’ professional experi ences is essen- 
tial to identify thefeatures that most effectively promote 
outcomes that address both student and teacher needs 
(Westerlund, Garcia, Koke, Taylor, & Mason, 2002). 
Such research can contribute to a clearerunderstanding 
of how learning environments interact with individual 
and group differences to optimize design of existing 
and future opportunities (Duke, 2004; Westerlund, et 
al, 2002). 

T hepresentstudyfol lowsacohort of 17 K -12 teach- 
ers through a six-week resident learning experience in 
science and engineering, and on into the planning and 
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implementation of applications for their classrooms. This Research Experiences for 
Teachers (RET) program was examined using the strategic approach of design-based 
research, with its fluid, adaptive management of the complexity of authentic learning 
in situ and its attentive documentation of expected and unexpected events, in process 
and products, to capture the richness of teachers' and mentors' experiences. 


Background 

The design-based research approach used in this study considered multiple 
elements of teacher I earning and transfer in order to obtain a rich and complex data 
set. Research on effective teacher professional development, adult learning, situ- 
ated cognition, and learning transfer were utilized to inform the evaluation design. 
Teachers' and mentors' perceptions of their experience were essential in this study 
and the data collection was structured to meaningfully include these factors. 

Teacher Professional Learning 

Effective teacher professional development should: (1) include well-defined 
theory in teaching and learning; (2) build in-depth knowledge and skills; (3) model 
strategies; (4) build learning community; (5) support teachers' leadership; (6) link 
to larger educational communities; and (7) be continually reassessed for improve- 
ments (Bel I & Gilbert, 1996; Bransford, Brown, & Cocking, 1999; Wilson & Berne, 
1999). In order to promote substantive changes in teaching practice, educators need 
opportunities to study both content and pedagogy (Berliner, 1991; Branscomb, 
1993; Wilson & Berne, 1999) and to engage actively in situations merging content 
with meaningful learning contexts (Bybee& Loucks-Horsley, 2000; Garet, Porter, 
Desimone, Birman, & Yoon, 2001). However, few teacher professional develop- 
ment opportunities integrate these important design features (Westerlund et al ., 
2002; Wilson & Berne, 1999). The RET program is designed to address many of 
these requirements by offering teachers an opportunity to participate in authentic 
research experiences and then translate them into classroom practice. 

During the six- week RET summer experience, teachers work in small groups 
of four to six with a university research mentor to answer testable questions related 
to specific engineering disciplines. They are asked to turn what they learn into 
classroom activities that will increase their students' understanding of concepts 
they regularly teach. The teachers are offered an opportunity to write proposals to 
receive funding for classroom materials related to the activities they create in order 
to encourage full participation. 

Teacher professional development couched in authentic field experience can 
promote know ledgeand ski 1 1 developmentforteachers, especially when collegial ity 
and communication between scientistsandteachersarehigh (Dresner, 2002; Dresner 
& Worley, 2006; Westerlund etal ., 2002). Summer institutes such astheRET, outside 
of teachers' daily professional contexts, offer in-depth learning opportunities and 
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promote collegial sharing (Basista& Mathews 2002; Keyser,1997). Participation 
in this type of professional community facilitates development of expertise and 
enculturation that supports innovation (H ardre, Ge, & Thomas, 2005; Gordon, 
2002). Collegial relationships also enableongoing development and application of 
skills, improve success expectations, and support teacher follow-through (Dresner 
& Worley, 2006; Hausman Si Goldring, 2001). The intent of the RET structure is 
to encourage ongoing teacher-teacher and teacher- mentor communication and 
collaboration to promote and sustain innovative teacher practices. 

Effective professional learning addresses teachers’ individual differences, 
beliefs, knowledge, and attitudes (van Driel, Beijaard, Si Verloop, 2001; Wilson Si 
Berne, 1999) and considers that learners’ investment of effort in learning depends 
on their perceptions (Salomon, 1984; Bransford, etal., 1999). Teachers are moti- 
vated by incentives, but also by interactions with like-minded colleagues, sharing 
their passion, experiences, and insights (Barnes, Hodge, Parker, Si Koroly, 2006; 
Dresner, 2002; van Driel, et al ., 2001). As adult learners, teachers are motivated 
to engage in professional development based on relevance (Barnes et al., 2006; 
Gilmer, 1997), and theopportunity to share ideas with professional peers (Hausman 
Si Goldring, 2001; Lieberman Si M iller, 1999). These elements of teacher I earning 
were considered in the RET evaluation design, and data was collected to assess 
teachers’ perceptions of the experience in these terms. 

Transfer of Skills and Authentic Experience 

The development and transfer of usable skills requires authentic experience 
(Brown, Collins, & Duguid 1989; Hutchings, 1991). Too often, however, school 
experiences fail to I ink to the authentic practices of professional cultures (Bransford 
etal., 1999; Putnam & Borko 2000). Teachers tend to select and integrate new tools 
and strategies piecemeal into their existing teaching practice (van Driel, Beijaard, 
& Verloop, 2001). Integrated, holistic, conceptual change requires extensive time, 
resources, and support (Posner, Strike, Hew son, & Hertzog, 1982), and isfacilitated 
in authentic, professional communities of practice (Dresner & Worley, 2006; Fish, 
1980; Little, 1993; Zellermayer& Tabak, 2007) such as those created by RET s.The 
cognitive dissonance generated in this type of transformative learning experience 
forces teachers to seek ways to modify their current practice (Loucks-Horsley, 
Love, Stiles, M undry, & Hewson, 2003). 

The ultimate measure of success in a program like RET is the extent to which 
teachers assimilate knowledge in a meaningful way and subsequently teach their 
students whatthey learned. In examiningtheissueofauthentictransferforthepresent 
study, wedistinguished two phases/typesofteacheroutcomes: ( 1) theauthenti city of 
theon-siteteacher learning experience, and (2) theteachers' translation and eventual 
transfer of the authenti c experi ence to the cl assroom . T eachers need scaffol di ng and 
support to successfully accomplish a transition-and-transfer task from their own 
learning experience to that of their students (Garet et al, 2001; Lieberman, 1995). 
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Without support in thetransition, they may experiencefrustration and helplessness 
and may even abandon their project implementation (Granger, 2002; Putnam & 
Borko, 2006). However, those who feel supported through the transition should 
retain their efficacy from the on-site experience and sustain their commitment to 
thetransfer-to-practicetask. Few studies of teacher research experiences have fully 
examined effects and implications through the implementation phase (Beighley, 
1998; Gonzales, 1998). 


Design-Based Research 

Design-based research (Brown, 1992; Collins, 1992) is the systematic design 
and subsequent implementation and study of instructional tools and strategies in 
authentic practice (Design Based Research Collective, 2003). Using empirical 
methods and theory-driven design of learning environments, its goal is to under- 
stand how, when, and why educational innovations work (or do not) in complex, 
authentic learning and performance contexts (Design Based Research Collective, 
2003; Hardre, 2003; Tessmer & Richey, 1997). The field of education needs new 
research approaches that address implementation in practice (National Research 
Council, 2002; Ponte & Smit, 2007) and lead to useful knowledge that informs 
practice (Lagemann, 2002; Shavelson & Towne, 2002). Design-based research 
ensures rigor and adherence to solid theoretical underpinnings with attention to the 
culture of context (Design Based Research Collective, 2003;Thornkil dsen, 2005), 
while documenting processes and products of adaptation (Wang& Hannafin, 2005; 
Yates, 2004). 


Research Questions 

The research questions for this study focus on teacher learning, engagement, 
and classroom implementation with respect to the program design. The intent is 
to examine factors that contribute to or diminish the effectiveness of the program 
and its desired outcomes in order to inform practice. 

1. What personal, interpersonal, and environmental factors contributed 
to teachers' engagement and other motivational characteristics, as well as 
their science knowledge and skill development? 

2. What factors contributed to the development and support of the educa- 
tional learning community in the on-site experience and afterward? 

3. What factors contributed to teachers' transfer and implementation of 
the science teaching strategies from their on-site mentoring to their own 
classroom teaching? 

4. What by-group differences existed that might illuminate findings related 
to these key outcomes? 
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Methods 

Participants 

Teacher-learners. Program participants were 17 K -12 teachers, from the fol- 
lowing subject areas: seven in math, nine in science, and one in special education. 
There were eight male and nine female; nine with bachelor's degrees only, seven 
with master's, and one with a Ph.D. Ten of the teachers had 1-5 years of teaching 
experience, three had 6-10 years, and four had more than 10 years. The 16 schools 
from which the teachers came had the following characteristics: levels— 11 high 
schools, five middle schools, one elementary school; locations— three rural, 14 
from two metropolitan areas. 

Faculty mentors. Three primary faculty mentors (all Ph.D.s), and their staffs 
(onejuniorfaculty member, plus four undergraduate lab assistants) worked within 
three labs. The labs were in three different engineering departments (Industrial 
Engineering, Computer Engineering, and Environmental Engineering) on two 
campuses of a research university in the Southwestern U nited States. 

Program Scope 

Thescopeof the present program included teacher recruitment and orientation, 
a six-week on-site (resident) university experience with mentor and peer learning 
communities, and support for transition and transfer of the learning experience to 
classroom application. Teachers were volunteer participants recruited from schools 
around the state. Teachers were asked to identify research issues and interests for 
classroom research and instructional innovation, and were selected for the pro- 
gram based on match of their personal project goals with the program’s goals and 
resources. Teachers were then placed in cohort-based mentoring groups for the 
on-site group experiences, based on alignment of their project goals with mentor 
expertise. G roups consi sted of 5-6 teachers i n each of the three mentori ng groups i n 
the three engineering labs: group 1, industrial engineering (IE); group 2, computer 
engineering (CE); and group 3, environmental engineering (EE). 

Following the early summer (M ay-J une) on-site experience, teachers returned to 
thei r home communiti es for the remai nder of the summer and f al I . E ach teacher w rote 
a proposal for classroom application of the skills and concepts learned and practiced in 
thecohort.Proposalsincludedrequestsforfundingfornecessaryequipmentand supplies 
required for implementation. Proposals were reviewed and evaluated by the mentors, 
andfeedbackgiven.Whenproposalswereapproved(andfunded),therequired materials 
were provided by the RET team. Teachers implemented their proposed projects, wrote 
up the results, and submitted those reports to the RET team, receiving feedback from 
thei rmentors.Teachershad ongoing access to theonlinediscussionsand communication 
tools and to their peers in the learning community throughout the 12-month program 
I ifecycl e. T he onl i ne access was developed as a secure si te i n M oodl e § an open-source 
information management system (IM S) software. 
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Data Scope and Collection 

Data for the study included multilevel indicators, collected from multiple 
sources over the full year of the program cycle, in forms both qualitative and quan- 
titative. Data were collected via various methods and media, in person and online, 
to maximize access, authenticity, and clarity, and to promote salience and proximity 
to experience. This approach also served to streamline analysis and promote data 
security. 

A shared, secure, virtual data col lection and communication system was created 
in M oodle 8 to provide a central access point for the multi event, year-long program 
(including links to questionnaires, discussion tools for community sharing, and 
product submission upload capabilities). Links to surveys, discussion prompts 
and document submission instructions were posted to the online site, and email 
reminders sent to participants. In addition to theonlinecoll ection of questionnaire 
and product data, observational and interview evidence was collected, viaface-to- 
face interactions, to ensure that the voices of participants were clearly heard and 
understood, and to enable conversation and dialog with and among participants. 

Instrumentation 

Evaluations of process and product indicators were made using a combination 
of standardized quantitative scoring rubrics and supplemental qualitative descrip- 
tors as evidence, to promote objectivity and comparability of judgments. Both the 
mentors (as active participants in the experience) and two evaluators (as objective 
others) scored and commented on the materials. Assessments included both well- 
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validated instruments used previously with similar groups, and new instruments 
contextualized for this project using assessment design best practices (e.g., Chatterji, 
2003; Reynolds, Livingson, & Willson, 2006; T hornki I dsen, 2005). The researchers 
soughtthisrangeasan appropriate balance of instrumentation for an applied, mixed- 
method project (e.g., C reswell, 2003; Creswell & Plano Clark, 2007; Fraenkel & 
Wallen, 2006). Instruments are described below, and additional technical details are 
included in a separate publication (H ardre, Slater, & Nanny, 2009). 

Individual and background characteristics (demographics). Background 
questionnaires on age, gender, race/ethnicity, years of teaching experience, years 
in current school, subject(s) taught, grade level(s), educational level, and licensure 
type were administered on day one of the on-site experience. Data were collected in 
digital form, also serving as an orientation to, and practice in, the online system. 

Evaluation of teacher participation and engagement. M entorevaluationsof teach- 
ers’ participation and engagement during the on-site research experience included a 
combination of measures. M entors used a standardized observation scoring rubric 
(7-item, L i kert-type, 5-poi ntscale, "N otatal I true" to "Very M uch true”), rating teach- 
ers’ demonstrations of participation and engaged behaviors, verbal and nonverbal, at 
two admi nistrations (weeks 3 and 6). To supplementthe numeric evaluations, mentors 
provided illustrative descriptive evidence on a page appended to each of the rubrics, 
plus independent interim notes and observations of student participation. M entors 
also reported individual and group participation and engagement in their interviews, 
near the end of the program period (weeks 27-30). 

Evaluation of teacher products. M entors and evaluators completed a set of 
rubrics and observation notes on products that teachers generated during the on- 
site experience (7-item, Likert-type, 5-point scale, "Not at all true” to “Very M uch 
true”). Though the specific activities and projects in groups varied by context, the 
rubrics featured generalizable characteristics of teacher quality of engagement that 
were recognizable in research products across contexts. The same scoring model, 
w i th ill ustrati ve descri pti ve evidence and interim notesand observations, was used 
as for participation. 

Teacher perceptions surveys. M ultiscale questionnaires were used to assess 
a range of present- and future-oriented teacher perceptions and intentions related 
to their learning experience and of the embedded skills and concepts they were 
learning. The 40 survey items were selected a priori to assess seven different 
constructs: perceived value, relevance/utility and benefit of the current skills; and 
future plans to use, efficacy in transfer of, feasibility, and perceived fit for thei r K -12 
classrooms. The constructs were selected based on their demonstrated importance 
in supporting transfer from teacher professional development. The 4-5 items for 
each subscale were presented with Li kert-type, 7-point scales (subscaleCronbach’s 
alphas .90-. 94). Questionnaires were administered twice (weeks 3 and 5). 

Online implementation planning and discussion. To bridge from the on-site 
resident experience to implementation and integration of the content, teachers 
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engaged in online implementation planning and collaborative discussion. Eight 
prompts elicited teachers’ thinking through conceptual and procedural aspects 
of the transfer and integration of the science and engineering content into their 
classrooms. 

Proposal documents. Teachers wrote a plan for an action research project, tak- 
ing their summer research experience into their classrooms. Proposal documents 
consisted of two parts: (1) the research plan, and (2) the intervention/lesson plans. 
The two parts focused on two key elements of teacher transfer: (1) designing the 
resea rch proj ect to generate meani ngf ul i nf ormati on on student I earni ng (transferri ng 
the scientific process); and (2) designing the science lesson to pass on principles 
of scientific knowledge and exploration to students (translating the teaching of 
science). Proposals were scored by mentors and evaluators using a standardized 
rubric (0-3 numeric scale). Criteria were aligned with the performance goals for 
the two outcomes, operationalized in the task specifications. The required score 
for proposal approval (and funding) was at least 20 (of 30 possible) by at I east two 
raters on both components. Teachers whose proposals did not meet this criterion 
were given feedback and invited to resubmit. 

Project implementation reports. After implementing the classroom research 
project, teachers wrote up their results and findings in a research implementation 
report and submitted it. Reports were evaluated using a rubric parallel to that used 
to evaluate the project proposals (0-3 numeric scale) and descriptivecriteria aligned 
with the report specifications (30 points possible). 

Periodic teacher reflective writing prompts. Over the course of the proj ect period, 
teachers were given periodic sets of online writing prompts timed appropriately for 
thephaseof the project in which they were currently engaged (or had just completed). 
Prompts addressed teachers’ perceptions of their own progress and conditions in their 
school environments relevant to project implementation and integration. Generative, 
open-ended items required narratives or analysis as responses. 

Email conversations. Though these were an informal, non-systematic source 
of data, the wealth of email that flowed among mentors, teachers and program staff 
provided critical insights about what was occurring in the program, particularly 
after teachers left the on-site resident experience. We collected a total of 147 
emails for analysis. 

Focusgroupsand interviews. Semi -structured individual and group conversations 
were conducted with participants at various points in the program lifecycle through 
i ntervention and implementation phases. Questions addressed perceptionsof learning 
in the RET experience, implementation, and school and community support. 

Analysis 

Data were analyzed in multiple ways for: (1) whole-group patterns, such as 
teachers’ needs and perceptions of their experiences, indicators of their develop- 
ment, and functional program implementation; and (2) by-mentor-group differences 
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that shed lighton design of experiences, support effectiveness, and overall program 
effects. M entor group differences were not viewed as right or wrong, but examined 
through the lens of best fit with program goals and outcomes. 

Analysis included a variety of methods, matched to the questions and data 
types. For the quantitative data wegenerated overall and by-group means in Excel. 
As the sample size (N =17) was inadequate to expect any statistical power or to 
meaningfully demonstratesignificance, scores were simply compared for differences 
by individuals and groups, and for patterns of change over time. For the qualitative 
data, multiple external evaluators independently coded the text of responses, and 
then the researchers compared them for patterns of meani ng, change, and perceptual 
differences. Toward the synthesis of findings, both types of data sources were used 
to address the research questions. They were analyzed by both mixed-method and 
multimethod approaches as appropriate to the questions, with multisource data 
triangulated to ensure validity and identify differences in stakeholder perspectives 
(see C reswell, 2003; Denzin & Lincoln, 2003; Fluberman & Miles, 2002). The 
findings reported below emerged as consistent across data sources and types, as 
relevant to each research question. 


Results 

Question I: Personal, Interpersonal, and Environmental Factors 

Our first research question was: "What personal, interpersonal and envi- 
ronmental factors contributed to teachers' engagement and other motivational 
characteristics, as well as science knowledge and skill development?" To address 
this question, we examined teacher perceptions questionnaires, reflective writing 
(of teachers and mentors), mentor evaluations (of participation and products), 
focus groups, and interviews. 

Learner engagement is important because engaged learners encode more 
knowledge, develop more expert skills, and are better prepared to transfer to ap- 
plication contexts (Druckman & Bjork, 1994; H ardre & M iller, 2006; Kytle, 2004). 
Engagement was a primary goal of the program specifications (National Science 
Foundation, 2007) and a key predictor of learning and professional skill develop- 
ment (Latham, 2007; Taylor, M arienau, & Fiddler, 2000). 

Overall, both teachers and mentors reported that teachers were engaged inthe 
RET experience and that they experienced an opportunity to increase their learning. 
The majority of teachers tied their engagement to connections between their current 
learning and their future teaching needs. M ost expressed some conflict between the 
authenticity of the university research experience and the utility of what they were 
learning for classroom transfer. Consistent with the theoretical linkages, teachers 
who found the most value and utility in what they were learning also reported the 
hi ghest engagement; and those w ho were most engaged had the strongest i ntenti ons 
to transfer and use the content i n thei r classrooms. T here were differences i n teach- 
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ers’ degrees and types of engagement, vestedness i n the process, and the extent to 
which valued outcomes were tied to engagement, as reflected in the comments on 
perception surveys and in interviews. 

Teachers identified diverse factors as influencing engagement. They acknowl- 
edged the i nfl uences of thei r past experi ences and future needs and expectations on 
their current perceptions and processing of the on-site research experience (e.g., "1 
feel that the entire project is designed to promote group learning, and the engineer- 
ing push also goes with my math and the group issues that I have had in the past."). 
They identified specific elementsof theexperiencethatpromoted intellectual skills 
and linked them to research tasks (e.g., "I am encouraged by the thinking skills I 
am practicing. The on-site research experience is teaching me to be more confident 
in my thoughts or ideas on the research topic"). 

Some teachers defined engagement in terms of utility of the content they were 
exposed to (e.g., "[I am] mostly trying to focus on how I can use this experience."). 
Others experienced personal engagement in the research but felt a sense of conflict 
with their reason for coming (e.g., "[I got] so engaged [in the research] that I forgot 
that I was here for my kids"); and one mentor confirmed that "some teachers got 
so involved in the research that they did not make the transfer to teaching." Some 
defined their engagement around the community and collaboration: 

I am definitely staying engaged because I find goals and objectives for our group 
to work on. I am a proactive individual and try to make sure I am contributing to 
my group's project and see how I might use the project in my own classroom. 

Others identified the research opportunity itself as engaging, including the col- 
laborative dynamic and spirit of inquiry: 

The research itself keeps me engaged. I enjoy discussing our progress and results 
with the lab techs and other teacher participants. I know we are the experiment 
and what we are doing is someone else's research. It seems like one big circle, 
where everything and everyone involves inquiry. 

Others cited the col I aborati on and peer-support features, along with the tools (e.g., 
"Our group has been very supportive. The sensors we have been using have also 
brought more ideas for other projects."). M ost teachers found the col I aboration and 
community experience particularly satisfying (e.g., "I havelearned a lot especially 
about collaboration. The best part is being able to work with other teachers"). 

A few teachers expressed the feeling that the work they were assigned to do 
was not personally meaningful and that they lacked ownership of the project tasks, 
that they "felt I ike lab aides". Some participants wanted more time to achieve their 
goals, while others thought that tasks took more time than they should; but both 
groups said they had learned much (e.g., "It has been a real pleasure working [and 
learning here], I feel I ike I will go back to the classroom and encourage my students 
to problem solve on a higher level"). 

Challenge is important because it is related to goal achievement, motivation, 
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learning, and perceptionsof the valueof instructional and developmental events.AII 
of the teachers reported that they felt challenged in their learning experience, but 
descri bed di ff erent I evel s and ty pes of chal I enge, and rel ated percepti ons. C hal I enge 
was often linked to the teachers’ perceptionsof difficulty in translating content and 
ski 1 1 sfrom the research experienceto classroom practice. Several teachers welcomed 
the personal challenge of lab science, but were concerned about the professional 
challenge of classroom transfer. Thus, they distinguished between personal and 
teaching goals regarding perceptionsof chal I enge (e.g., "I am definitely challenged 
[but] It is going to be hard to implement it in my classroom.”). Similar perception 
with a different strategy emerged in this teacher’s comment: "I am challenged with 
the [university] research and I in turn challenge myself to see how I will apply what 
I have learned back into the classroom.” Another perspective was distinguishing 
between the content and context as sources of intellectual and personal or social 
challenge (e.g., "I do feel challenged in more than one way. The research topic is 
challenging my brain, and working with three other people creates another set of 
challenges but good ones”). 

Teachers enthusiastically said that they were learning, but generally tied their 
learning more to thinking about classroom use than to being engaged in the current 
(more pure, traditional, or laboratory) research. This distinction indicates that the 
teachers are divided between focusing on the utility versus the authenticity of their 
on-site research experience. They appear to regard research skills as discrete vs. 
integrated (both in current experience and in their classroom instruction). Teachers' 
conceptualizations of authenticity and challenge result in different perceptions of 
the transfer of their current learning to classroom research, with those differences 
ranging from proximal (near and similar) to distal (far and different). Some saw 
transfer to their classrooms as easy, based on interpreting their own experience as 
aligned with and closely translatable to the experience they wanted to create for 
their students (e.g., "Yes, I feel like doing science is a great way of learning sci- 
ence. The on-site research experience gives teachers a true science experience that 
they can then take back and work into their classrooms”). While some found the 
experience mostly focused on the lab research and further removed from learning 
to teach science (e.g., "The experience continually focused me towards learning 
research techniques. I had to keep reminding myself of my students.”), others found 
the opposite true and sought more "real” research (e.g., " I feel I ike what we have 
accomplished can transfer to our classrooms easily. I do notfeel likewehavedone 
'real research' though”). 

On changes that indicated their learning, the teachers said they shifted from 
thinking about the "steps with lab” to the "problem-solving steps” (from the primar- 
ily technical, mechanical, or procedural to more substantive and deeper elements 
of tasks). M entorsalso independently reported the same shift, saying said that they 
saw learners’ questions shift from "questions on technical and mechanical issues” 
to "real inquiry, self-questioning, and scientific skepticism.” 
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Expectations provide a framework for organizing and responding to new ex- 
periences, and goals that arise from expectations are a lens through which learners 
view educational opportunities. How teachers link the research experience to their 
expectati ons of the R E T experi ence form a cri ti cal mass that emerges as i nfl uenti al 
on their engagement, learning, and satisfaction with the program. These links are 
also related to their perceptual responses to all of the opportunities they are af- 
forded. The teachers’ perceptions of theirlearning and engagement are tied directly 
to their expectations of the program, from a range of tangible and intangible tools 
to more esoteric goals of traditional academic research. 

Overall, on Question 1, teachers’ motivation and subsequent engagement were 
supported by their perceived relevance, value, and transfer utility of the content. 
These perceptions were based on their past experience and future needs, which 
influenced how they processed their mentor’s activities, style, and communication. 
They were also influenced by level of intellectual and personal challenge provided, 
and how they defined transfer goals and expectations. 

Question 2: Development and Support of Learning Communikty 

Oursecond research question was: "Whatfactorscontributedtothedevelopment 
and support of theeducational learning community in the on-si teexperi ence and af- 
terward?" To address thisquesti on, weexamined teacher perceptions questionnaires, 
reflective writing (of teachers and mentors), mentor evaluations (of participation 
and products), focusgroupsand interviews, proposalsand reports. Teacher-mentor 
support, trust, and availability are no less important for adult teacher- learners than 
f or thei r ow n K - 12 students (B rookfi el d, 2006; M erri am, C aff arel I a, & B aumgartner, 
2007). Consistent with findings from previous studies (e.g., Barnes et al., 2006; 
Brown & M el ear, 2007; Dresner& Worley, 2006; Westerlund etal, 2002), teachers 
identified a number of factors that influenced their development of community. 

Teachers developed community based on both similarities and differences. 
Their shared interest in science I earning and teaching created "common interests” 
and "common ground” for communication and collaboration. The immersive na- 
ture of the on-site experience including the residency component ("living, eating, 
sleeping together”) promoted "knowing each other well,” building "closeness,” 
and community. Facing the demands of challenging tasks caused groups to "learn 
and depend on each other’s strengths” and develop mutual respect, as members of 
cohorts took leadership on various (e.g., mathematical, mechanical or technologi- 
cal) components of complex projects. 

A particularly interesting differenceemerged between theteachers’ and mentors’ 
perceptions of the scope of the program, and the attention mentors gave teachers 
after they left the on-site phase, in the teachers’ view, the program continued into 
the school year as they sought to implement their proposals, while the mentors saw 
their role as minimal beyond the summer experience. Teachers felt less connected 
and supported in the off-site phase than when on site (e.g., "felt alone” and "had 
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trouble reconnecting with mentors and peers"). The reduction in contact intensity 
and focus for the faculty mentors seemed to create a critical disconnect for teach- 
ers in the adequacy of the support they felt they were receiving, in the integration 
and networking phases. They "expected more connection” after going home (back 
to school) and "felt less successful" because they lacked support that they needed 
and wanted. The most challenging goals of the program were after the teachers 
left, integrating RET into their classrooms and building networks, for the teachers 
to continue integration and implementation in their schools. This is when teachers 
need their mentor and peer community of practice most profoundly, to extend and 
apply the knowledge they had built together. 

M entor avai I abi I i ty and support were sensitive and influential. Consistentwith 
previous research (e.g., Barnes et al, 2006), more than one teacher said that the 
mentor was very expert and credible but seemed "too busy." Teachers with this 
perception reported less enthusiasm for the program overall, in perceived learn- 
ing during the on-site experience, and in expectations of success afterward; and 
these teachers were less successful in implementation than those who perceived 
their mentors as accessible and involved. This critical perception underscores the 
importance of mentor attention as an aspect of teacher immersion experiences, with 
the perceived support, access, and availability of mentors to their teacher-1 earners 
emerging as critical elements in teachers’ overall program success. Ongoing com- 
munication among mentors and teachers was essential to building and sustaining 
networks for implementation. 

Overall, on Question 2, teachers’ ongoing learning community was developed 
and supported primarily by their interactions during the on-site experience and 
the ease of communication and interactions (with mentors and peers) afterward. 

1 nitial and continued mentor support and communication were critical to commu- 
nity development, so it was essential to explicitly provide user-friendly, accessible 
communication tools for teachers to sustain community interaction. 

Question 3:Transfer and Implementation of Teaching Strategies 

0 ur thi rd research question was: "W hatfactors contri buted to teachers' transfer 
and implementation of the science teaching strategies from their on-site mentoring 
to their own classroom teaching?" To address this question, we examined periodic 
questionnaires, reflective w riti ng, email conversations, interviewsandfocusgroups, 
proposals and reports, and evaluations of proposals and reports. 

in data from teachers and mentors, linkages to classroom integration differed 
widely. Perceptions linked to implementation in classrooms also differed both in 
strength and stability by group (see details in by-group differences). However, 
overall, teachers demonstrated good strength and consistency in their responses 
to the professional development activities, with the foil owing means for subscales 
(out of 7): value 5.45; utility 5.3; benefits 5.25; feasibility 5.4; fit 5.15; confidence 
in implementing 5.25; and plan to use 5.5. These perceptual scores reflecton moti- 
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vational issues (e.g., perceived value, utility and benefit), and on potential transfer 
to the classroom (via the future-focused characteristics of confidence, feasibility, 
fit, and plan to use). The items in this questionnaire target how they perceived the 
content and skills they learned, as useful, feasible, and beneficial for transfer to 
their own K -12 teaching. Teachers who used the online support system and support 
staff made a smoother transition to implementation and integration. Those whose 
mentors communicated the value of the system were more likely to adopt and use 
it, give feedback, and engagein collegial community as well. 

M ost of theteachers reported that they were successful i n transf erri ng pri nci pi es 
and strategies from RET to their classrooms. They were able to identify specific 
ways that they transferred learning from the on-site experience and implemented 
these in their K -12 classes. Only about half of the teachers (8 of 17) completed 
their full, planned classroom research projects (implemented the planned lessons; 
measured, analyzed and reported results). These were the most successful RET 
participants. M ost others implemented key features from RET and were able to 
discuss the perceived effects and benefits generally, so we considered them also to 
a degree successful in transferring the content and skills to their classrooms. 

As to influences on success in integration, several points are clear. Teachers 
emphasized the goal of integration from the first days of the on-site RET, through 
their proposal writing and implementation reports. Specific elements of the RET 
experience that they integrated most fully were consistent with what they valued 
and enjoyed when they were learners in RET and what they identified as having 
highest perceived utility for their own teaching. Some teachers identified elements 
of the general research experience, such as inquiry, discussion and problem-solv- 
ing analysis, that they are more attentively and consciously integrating into their 
classrooms afterRET than they did priorto it (e.g., "I use the research process and 
inquiry based teaching more”). Other teachers identified very specific changes in 
their design of daily activities that they attributed to RET, such as using collabora- 
tion in the research process: 

Each section taught contains daily demonstration explaining the object of the 
period, toward the end of the sections students are required to break into small 
groups, 3 or4studentgroups, develop a research question, determinea hypothesis, 
laboratory testing, write an evaluation of their results. 

M any teachers shared that they integrated strategies from RET into their general 
teaching practice, not just into the implementation of their planned projects (e.g., 
“I do more of the real life problem solving. We also talk more about applying our 
math"). Some teachers shared otherfeaturesthatthey planned to implementin their 
classes, based on what they found useful during the laboratory research experi- 
ence (e.g., "... the importance of randomizing the experiment so that results were 
not biased to a learning curve. The reason this helped me so much was to receive 
val i d results from my experi ment" ) . T hese were exam pi es of direct strategy transfer 
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(proximal transfer of what they learned in RET). Beyond the science, RET mentors 
modeled important interpersonal qualities that theteachers adopted and attribute to 
their mentors' examples, from modeling rather than telling (e.g., "The main thing 
that my mentor passed along to me was the need to develop closer relationships 
with my students so that they will approach me with questions/ideas”). 

Overall, on Question 3, thecommunity development and continued communi- 
cation wereimportantsupportsforimplementation and integration in classrooms. 
M entor relatedness and modeling also proved critical, as teachers tended to take 
back and transfer what they saw modeled by their mentors in both content- related 
and more general teaching strategies. 


Question 4: By-Group Differences 

Our fourth research question was, "What by-group differences exist that may 
illuminate findings related to these key outcomes?" To address this question, we 
examined all of the data sources, from teachers and mentors, divided into clusters 
by mentoring groups. 

By-Group Differences. M arked differences in overall design of the learning 
opportunity by mentoring group emerged in the descriptions from both mentors 
and teachers. By-group differences in the patterns of the teacher outcomes data 
(both quantitative and qualitative) underscore the importance of examining these 
features. Table 2 compares features of the three mentoring groups in their overall 


Table 2 

Group Design Features 



Group 1 

Group 2 

Group 3 

Research Area 

Industrial Engineering 

Computer Engineering 

Environmental Engineering 

Mentor Contact 

• Junior faculty member - first contact 

• Moderate presence - high availability 

• Active collaboration - projects & class 
activities 

• Provided structured face-time 

• Direct contact time - 4 hours/week 

• Mentor - first contact 

• High presence - daily engagement 

• Active collaboration - projects & class 
activities 

• Provided structured face-time 

• Direct contact time - 20 hours/week 

• Graduate research students - first 
contact 

• Moderate presence - low availability 

• High engagement on core projects and 
class activities 

• Low structure 

• Mentors - 4 hours/week (GAs 20/wk) 

Group 

Collaborations 

• Whole group collaboration 

• Mentor and within group leadership 

• Collaboration on project work 

• Groups mixed across tasks and projects 

• Collaborative group tasks not assigned 

• Mentor leadership 

• Integrated structure and questioning 

• Focused on individual core projects 

• Two self-selected groups 

• Group leadership with mentor support 

• Individual groups independent research 

• Groups had own research projects 

Task Autonomy 

• Low independence and low autonomy 

• Low independence and low autonomy 

• High independence and high autonomy 

Scaffolding of tasks 

• High scaffolding of pre-planned 
research tasks provided by mentor 

• Some tasks related to mentor’s current 
research 

• Structured learning with multiple 
problem-solving activities by mentor 

• Tasks not specifically related to 
mentor’s current research 

• Emphasized low structure of learning 
tasks and high independent learning 

• Research directly related to mentor's 
current research 

Authenticity 

• Low traditional authenticity 

• Authentic practice linked to classroom 
applications and examples 

• Low traditional authenticity 

• Authentic practice linked to classroom 
applications including tools 

• High traditional authenticity 

• Traditional academic/scientific 
research environment 

Transfer 

• Explicit scaffolding of transfer using 
activities with direct classroom 
application 

• Involved hands-on tools and activities 
appropriate to the K-12 classroom 

• Used focused questions designed to 
implicitly scaffold transfer of learning 

• Utilized tools appropriate to the K-12 
classroom 

• Low scaffolding of transfer - little 
explicit support for transfer 

• K-12 classroom tools not used - 
provided “cutting edge” research tools 
for teacher learning 

Integration 

• Explicit support of integration 

• Explicit support of integration 

• Implicit support of integration 
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learning environment design, mentor style and interactions, and differences in 
learner perceptions and behaviors. The mentoring groups are described in terms 
of autonomy (the amount of direct mentor supervision on research tasks), scaf- 
folding (the extent to which the mentor determined what the research tasks would 
be), traditional authenticity (the extent to which the research questions and tasks 
were related to the mentor's current research), and support (the amount of direct 
assistance the teachers were given for transfer and integration). 

Perceptual and Effects Differences. Teachers in the different groups reported 
very different perceptions of their learning experiences overall, their mentors as 
teacher-facilitators, the perceived linkages to classroom teaching, and the perceived 
utility, feasibility, and fit with classroom teaching. These differences in perceived 
patterns parallel the group divisions and other prog ram- relevant outcomes. W hat 
emerged, not by explicit design but by the mentors' choice, was an exceptionally 
rich intervention-comparison design-based experimentacrossthethreegroups, on 
degree of: (1) structure and direction vs. fluidity and independence; (2) explicit 
support of transfer to classroom; and (3) degree and nature of collaboration. These 
features are summarized in Table 3. 

Overall, for Question 4, by-group differences in goals, structure, mentor pres- 
ence, and explicit scaffolding of transferand integration apparently exerted profound 
effects on learner outcomes. The on-site learning experience and mentor-teacher 
interactions within it and beyond need to be closely examined and strategically 
designed to align with program goals, it can not be assumed that all mentors and 


Table 3 

Perceptual and Effects Differences 



Group 1 

Group 2 

Group 3 

Group Profile 
Characteristics 

• 3 M, 2 F; avg age 34; avg 3 yrs tchng 
exper; subjects- 3 math, 1 science, 1 
special ed.; 3 Masters, 2 BS degrees 

• 3 M, 3 F; avg age 47; avg 9 yrs tchng 
exper; subjects - 3 math, 3 science; 3 
Masters degrees, 2 BS degrees, 1 PhD 

• 2 M 4 F; avg age 33; avg 6 yrs tchng 
exper; subjects - 1 math, 5 science; 1 
Masters degree, 5 BS degrees 

Teachers’ perceptual and effects differences 

Overall Learning 
Experiences 

• Good classroom transfer, but desired 

more authenticity (“real science”) 

• Owned research and collaborative work 

• High level learning useful skills 

• Good classroom transfer , but desired 

more authentic research experience 

• Owned research and collaborative work 

• High level learning teaching strategies 

• Less successful transfer, but traditional 

research experience 

• Learned interesting research & lab skills 

• Less ownership but “authentic" research 

Mentors as 
teacher-facilitators 

• High interpersonal skills and high support 

• High interpersonal skills and high 
credibility 

• High interpersonal skills but less-than- 
optimal support 

Links to teaching 

• Linkages apparent and feasible 

• Linkages apparent, required time 

• Linkages less certain & accessible 

Utility, feasibility 
and fit with 
classroom teaching 

• High perceived utility, feasibility, fit 

• Modest confidence for transfer 

• High perceived utility, feasibility, fit 

• Highest confidence for transfer 

• Lower perceived utility, feasibility, fit 

• Lower confidence for transfer 

Mentors’ perceptual and effects differences 


Group 1 

Group 2 

Group 3 

Definition of 
“authentic” & 
strategy 

• “When teachers experience the critical 
components for themselves [so] we will 
not spend time on the research 
hypothesis development phase, which 
is where they bogged down last year.” 

• When “teachers see its utility for their 
classrooms, based on their needs and 
goals [so] I brought tools in to help 
them see how they could. . . give their 
students a more authentic experience.” 

• ” if they experience what we do in the 
university lab when w e do research 
[so] I brought them into my world and 
showed them how we do research . . . 
how to think like scientists.” 

Reciprocal 
learning and 
Adaptation 

• Translated & transferred strategies 

learned from teachers to undergraduate 
engineering teaching 

• Observed teachers using equipment, 
developed understanding of their 
methods, drew on their examples 

• Listened as teachers shared ideas & 
stories, adapted ways of illustrating 
concepts for them 
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teacherswill interpret program outcomes similarly. Such interpretations should be 
examined at the outset of such programs, to support consistent learning, develop- 
ment and satisfaction. 


Discussion 

Teachers overall enjoyed the RET experience; they appreciated the faculty 
expertise and insights, and admired their mentors. The teachers used desirable 
terms (terms that match program vision) for targets of research, including terms 
like "proper science," "real world," and "true science." Some perceived the science 
as "over my head" but acknowledged that it was a good way to be challenged, "to 
stretch myself." They recognized that there were multiple levels of learning occur- 
ring: “The real lesson [was not] lab process but the process of discovery." 

Every participant reported gaining something of value from the prog ram. Col- 
laboration with mentors and peers was a unifying theme across all target outcomes 
and stood out as a key benefit for the participants in the program. Communication 
and support were key factors in promoting program outcomes overall, all differ- 
ences considered. Those teachers who felt communicated with and supported in 
the RET experience were the most successful in learning and applying concepts 
and skills in the on-site context. Teachers made changes in both formal scientific 
process and daily classroom strategies, which they attributed directly to their RET 
mentoring. Those teachers who feltsupported and enabled i n thei r schools felt the 
most confident about fol lowi ng through to implementation and integration. Those 
teachers who saw relevance between their mentored activities and their own class 
content and goals selected applicable tools from the on-site context and transferred 
those toolsand principles to theirclassroomsthrough implementation most readily. 
The implementation of web-accessible technology for communication served an 
important purpose in bridging the distance gap, particularly through the transition 
to implementation in the schools. These findings underscore the importance and 
utility of mentoring with communication and supportfor learning and teaching as 
a model for effective teacher professional development. 

Teachers came expecting very different things from the program, and strong 
relationships were evident between teacher expectations and value gained, tied to 
the teachers' expectations of what the program was supposed to provide them. M ost 
participants struggled to arrive at a clear idea of the expectations of the program 
in order to construct personal meaning for themselves. An important difference in 
expectations was the question of whose responsibility it was to support strategic 
transfer to the classroom (mentors or teachers). It is cl ear that program-level goals, 
design of the research experience, and mentoring style interacted with teachers' 
expectations and individual differences. These findings underscore the critical role 
that perceptions and expectations play in effectiveness of teacher education and 
professional development opportunities. 
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The question of defining authenticity is an important one, and not easily 
answered; nor is there one best way to bridge between the two "worlds" of K -12 
teachers and research scientists. Both teachers and mentors independently defined 
authenticity at various places along a continuum, from authenticity being "more 
like real, university lab science” (bringing the teachers into "our world") to it be- 
ing "more like what teachers will do in their own classrooms" (taking our ideas 
and principles into "their world"). Some teachers wanted "pure research" but got 
praxis; others wanted praxis or explicit transfer, but got "pure research" ("cutting 
edge research", "authentic research experience"). W hat was "authentic" to teach- 
ers did not always match what was authentic to mentors, and such differences in 
definitions implicitly influenced the strategy use of mentors and the responses of 
teachers in this program. We could (consistent with much of the research literature 
and with National Science Foundation guidelines) define authentic research as 
research done the way it is done in the university, in a cutting edge, fully-equipped 
laboratory, supported by RAs and mentored by a research scientist. 

I f we accept that i t i s authenti c research w hen the experi ence si mul ates the way 
university scientists do research in the lab, then these teachers perceived authentic 
research as difficult, unfamiliar or foreign, and disconnected from their transfer 
needs. Alternately, if we define "authentic" as science the way teachers will use 
the ideas, processes and principles in their own classes, the teachers perceived it as 
clear communication and applicable to their needs (but some lamented the lack of 
"real research"). To distinguish between these two interpretations of authenticity, 
we might reframe the latter as authentic transfer, asK-12 teachers taking principles 
from the university laboratory to their own classrooms and successfully doing an 
appropriately-contextualized version of lab or field science, utilizing the mentored 
principles and concepts, so their own students learn what they did. 

Transl ati ng from authenti c research to authenti c transfer is diffi cultfor teachers 
on their own, and arguably they need access to both to be able to carry out both the 
on-site and off-site components of a program of this kind. However, if the mentors 
spoon-fed teachers their classroom strategies, if they scaffolded them too much, 
would they be being robbed of the opportunity and challenge to create, to discover, 
even to have an authentic research experience at all? These questions bear further 
exami nation, w ith i mpl ications across teacher professi onal dev el opment programs. 
The contrasts in by-group design parameters and effects raise a host of questions 
about the relationships among teacher differences, expectations, perceptions, 
learning and transfer, questions with implications for ongoing research in teacher 
professional development experiences. 

in this study we see three parts of teacher professional development. The first 
is know I edge and skills, the intangible tools, cognitive and psychomotor, to do the 
research tasks. The teachers were given these in the on-site experience. The second 
is equipment, the physical resources and tangible tools with which to do the work. 
Thesewereprovided throughthefunding proposals.T hethi rd is empowerment— in 
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self- perceptions, motivations, and drive necessary to carry out the plans, provided 
through initial and ongoing support and communication. 

M embers of two separate but related communities of practice werejoined in 
one shared discourse community. Their different perceptions, understandings, and 
vocabulary were critical to the success of their learning, development and transfer. 
A subtext of the teachers’ learning was the integration of the two communities in 
one discourse, not only one group teaching the other, but a blending and integra- 
tion of their ways of knowing. Teacher-learners, expert in K -12 science education, 
learned from university mentors, expert in engineering. Engineer-mentors learned 
about K -12 science education and the demands of effective teacher development. 
The result was a change in the practice of both groups of teachers, in K -12 educa- 
tion, in undergraduate engineering (reported by the mentors), and in the adaptive 
revision of the professional development program itself. The dynamic interaction 
framed on design-based research supported examination of internal processes that 
underlie success in authentic educational experiences for teachers. These findings 
can richly inform research and teacher professional development, open doors for 
inquiry, and illuminate strategic design. 
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